LB Scalability: Achieving the Right Balance Between Being Stateful and Stateless

A high performance Layer-4 load balancer (LB) is one of the most important components a cloud service infrastructure. Such an LB uses network and transport layer information for deciding how to distribute client requests across group servers. crucial requirement stateful per connection consistency (PCC); namely, that all packets same will be forwarded server, as long server alive, even if pool servers or assignment function changes. The challenge in designing throughput, low latency solution also scalable. This paper proposes highly scalable LB, called Prism, implemented using programmable switch ASIC. As far we know, Prism first reported can process millions connections second hundreds total, while ensuring PCC. due fact forwards hardware, during changes, while avoiding need maintain hardware state every active connection . We prototype proposed architecture showed scale 100 million simultaneous connections, accommodate more than update second.